Wiener Filter
   HOME

TheInfoList



OR:

In
signal processing Signal processing is an electrical engineering subfield that focuses on analyzing, modifying and synthesizing '' signals'', such as sound, images, and scientific measurements. Signal processing techniques are used to optimize transmissions, ...
, the Wiener filter is a filter used to produce an estimate of a desired or target random process by linear time-invariant ( LTI) filtering of an observed noisy process, assuming known stationary signal and noise spectra, and additive noise. The Wiener filter minimizes the mean square error between the estimated random process and the desired process.


Description

The goal of the Wiener filter is to compute a statistical estimate of an unknown signal using a related signal as an input and filtering that known signal to produce the estimate as an output. For example, the known signal might consist of an unknown signal of interest that has been corrupted by additive
noise Noise is unwanted sound considered unpleasant, loud or disruptive to hearing. From a physics standpoint, there is no distinction between noise and desired sound, as both are vibrations through a medium, such as air or water. The difference aris ...
. The Wiener filter can be used to filter out the noise from the corrupted signal to provide an estimate of the underlying signal of interest. The Wiener filter is based on a
statistical Statistics (from German: '' Statistik'', "description of a state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data. In applying statistics to a scientific, industr ...
approach, and a more statistical account of the theory is given in the minimum mean square error (MMSE) estimator article. Typical deterministic filters are designed for a desired
frequency response In signal processing and electronics, the frequency response of a system is the quantitative measure of the magnitude and phase of the output as a function of input frequency. The frequency response is widely used in the design and analysis of s ...
. However, the design of the Wiener filter takes a different approach. One is assumed to have knowledge of the spectral properties of the original signal and the noise, and one seeks the linear time-invariant filter whose output would come as close to the original signal as possible. Wiener filters are characterized by the following: # Assumption: signal and (additive) noise are stationary linear
stochastic process In probability theory and related fields, a stochastic () or random process is a mathematical object usually defined as a family of random variables. Stochastic processes are widely used as mathematical models of systems and phenomena that ap ...
es with known spectral characteristics or known
autocorrelation Autocorrelation, sometimes known as serial correlation in the discrete time case, is the correlation of a signal with a delayed copy of itself as a function of delay. Informally, it is the similarity between observations of a random variable ...
and
cross-correlation In signal processing, cross-correlation is a measure of similarity of two series as a function of the displacement of one relative to the other. This is also known as a ''sliding dot product'' or ''sliding inner-product''. It is commonly used f ...
# Requirement: the filter must be physically realizable/
causal Causality (also referred to as causation, or cause and effect) is influence by which one event, process, state, or object (''a'' ''cause'') contributes to the production of another event, process, state, or object (an ''effect'') where the ca ...
(this requirement can be dropped, resulting in a non-causal solution) # Performance criterion:
minimum mean-square error In statistics and signal processing, a minimum mean square error (MMSE) estimator is an estimation method which minimizes the mean square error (MSE), which is a common measure of estimator quality, of the fitted values of a dependent variable. In ...
(MMSE) This filter is frequently used in the process of
deconvolution In mathematics, deconvolution is the operation inverse to convolution. Both operations are used in signal processing and image processing. For example, it may be possible to recover the original signal after a filter (convolution) by using a deco ...
; for this application, see Wiener deconvolution.


Wiener filter solutions

Let s(t+ \alpha ) be an unknown signal which must be estimated from a measurement signal x(t). Where alpha is a tunable parameter. \alpha > 0 is known as prediction, \alpha = 0 is known as filtering, and \alpha < 0 is known as smoothing (see Wiener filtering chapter of for more details). The Wiener filter problem has solutions for three possible cases: one where a noncausal filter is acceptable (requiring an infinite amount of both past and future data), the case where a
causal Causality (also referred to as causation, or cause and effect) is influence by which one event, process, state, or object (''a'' ''cause'') contributes to the production of another event, process, state, or object (an ''effect'') where the ca ...
filter is desired (using an infinite amount of past data), and the
finite impulse response In signal processing, a finite impulse response (FIR) filter is a filter whose impulse response (or response to any finite length input) is of ''finite'' duration, because it settles to zero in finite time. This is in contrast to infinite impulse ...
(FIR) case where only input data is used (i.e. the result or output is not fed back into the filter as in the IIR case). The first case is simple to solve but is not suited for real-time applications. Wiener's main accomplishment was solving the case where the causality requirement is in effect; Norman Levinson gave the FIR solution in an appendix of Wiener's book.


Noncausal solution

:G(s) = \frace^, where S are spectral densities. Provided that g(t) is optimal, then the
minimum mean-square error In statistics and signal processing, a minimum mean square error (MMSE) estimator is an estimation method which minimizes the mean square error (MSE), which is a common measure of estimator quality, of the fitted values of a dependent variable. In ...
equation reduces to :E(e^2) = R_s(0) - \int_^ g(\tau)R_(\tau + \alpha)\,d\tau, and the solution g(t) is the inverse two-sided
Laplace transform In mathematics, the Laplace transform, named after its discoverer Pierre-Simon Laplace (), is an integral transform that converts a function of a real variable (usually t, in the '' time domain'') to a function of a complex variable s (in the ...
of G(s).


Causal solution

:G(s) = \frac, where * H(s) consists of the causal part of \frace^ (that is, that part of this fraction having a positive time solution under the inverse Laplace transform) * S_x^(s) is the causal component of S_x(s) (i.e., the inverse Laplace transform of S_x^(s) is non-zero only for t \ge 0) * S_x^(s) is the anti-causal component of S_x(s) (i.e., the inverse Laplace transform of S_x^(s) is non-zero only for t < 0) This general formula is complicated and deserves a more detailed explanation. To write down the solution G(s) in a specific case, one should follow these steps: # Start with the spectrum S_x(s) in rational form and factor it into causal and anti-causal components: S_x(s) = S_x^(s) S_x^(s) where S^ contains all the zeros and poles in the left half plane (LHP) and S^ contains the zeroes and poles in the right half plane (RHP). This is called the Wiener–Hopf factorization. # Divide S_(s)e^ by S_x^(s) and write out the result as a
partial fraction expansion In algebra, the partial fraction decomposition or partial fraction expansion of a rational fraction (that is, a fraction such that the numerator and the denominator are both polynomials) is an operation that consists of expressing the fraction as ...
. # Select only those terms in this expansion having poles in the LHP. Call these terms H(s). # Divide H(s) by S_x^(s). The result is the desired filter transfer function G(s).


Finite impulse response Wiener filter for discrete series

The causal
finite impulse response In signal processing, a finite impulse response (FIR) filter is a filter whose impulse response (or response to any finite length input) is of ''finite'' duration, because it settles to zero in finite time. This is in contrast to infinite impulse ...
(FIR) Wiener filter, instead of using some given data matrix X and output vector Y, finds optimal tap weights by using the statistics of the input and output signals. It populates the input matrix X with estimates of the auto-correlation of the input signal (T) and populates the output vector Y with estimates of the cross-correlation between the output and input signals (V). In order to derive the coefficients of the Wiener filter, consider the signal ''w'' 'n''being fed to a Wiener filter of order (number of past taps) ''N'' and with coefficients \. The output of the filter is denoted ''x'' 'n''which is given by the expression :x = \sum_^N a_i w -i. The residual error is denoted ''e'' 'n''and is defined as ''e'' 'n''= ''x'' 'n''nbsp;− ''s'' 'n''(see the corresponding block diagram). The Wiener filter is designed so as to minimize the mean square error ( MMSE criteria) which can be stated concisely as follows: :a_i = \arg \min E \left \right_.html" ;"title="^2 \right ">^2 \right where E cdot/math> denotes the expectation operator. In the general case, the coefficients a_i may be complex and may be derived for the case where ''w'' 'n''and ''s'' 'n''are complex as well. With a complex signal, the matrix to be solved is a
Hermitian {{Short description, none Numerous things are named after the French mathematician Charles Hermite (1822–1901): Hermite * Cubic Hermite spline, a type of third-degree spline * Gauss–Hermite quadrature, an extension of Gaussian quadrature m ...
Toeplitz matrix In linear algebra, a Toeplitz matrix or diagonal-constant matrix, named after Otto Toeplitz, is a matrix in which each descending diagonal from left to right is constant. For instance, the following matrix is a Toeplitz matrix: :\qquad\begin a & b ...
, rather than symmetric
Toeplitz matrix In linear algebra, a Toeplitz matrix or diagonal-constant matrix, named after Otto Toeplitz, is a matrix in which each descending diagonal from left to right is constant. For instance, the following matrix is a Toeplitz matrix: :\qquad\begin a & b ...
. For simplicity, the following considers only the case where all these quantities are real. The mean square error (MSE) may be rewritten as: :\begin E \left \right_.html" ;"title="^2 \right ">^2 \right &= E \left (x[ns[n.html"_;"title=".html"_;"title="(x[n">(x[ns[n">.html"_;"title="(x[n">(x[ns[n^2_\right_.html" ;"title="">(x[ns[n.html" ;"title=".html" ;"title="(x[n">(x[ns[n">.html" ;"title="(x[n">(x[ns[n^2 \right ">">(x[ns[n.html" ;"title=".html" ;"title="(x[n">(x[ns[n">.html" ;"title="(x[n">(x[ns[n^2 \right \ &= E \left [ x^2 \right ] + E \left \right_.html" ;"title="^2 \right ">^2 \right - 2E[x[n]s[n\\ &= E \left -i\right)^2\right_.html" ;"title="\left ( \sum_^N a_i w -i\right)^2\right ">\left ( \sum_^N a_i w -i\right)^2\right + E \left \right_.html" ;"title="^2 \right ">^2 \right - 2E\left sum_^N_a_i_w[n-i_\right_.html" ;"title="-i.html" ;"title="sum_^N a_i w[n-i">sum_^N a_i w[n-i \right ">-i.html" ;"title="sum_^N a_i w[n-i">sum_^N a_i w[n-i \right \end To find the vector [a_0,\, \ldots,\, a_N] which minimizes the expression above, calculate its derivative with respect to each a_i :\begin \frac E \left \right_.html" ;"title="^2 \right ">^2 \right &= \frac \left \ \\ &= 2E\left \left_(_\sum_^N_a_j_w[n-j\right_)_w_-i\right_.html" ;"title="-j.html" ;"title="\left ( \sum_^N a_j w[n-j">\left ( \sum_^N a_j w[n-j\right ) w -i\right ">-j.html" ;"title="\left ( \sum_^N a_j w[n-j">\left ( \sum_^N a_j w[n-j\right ) w -i\right - 2E [w[n-i]s[n \\ &= 2 \left ( \sum_^N E [w[n-j]w -i] a_j \right ) - 2E [ w[n-i]s[n \end Assuming that ''w'' 'n''and ''s'' 'n''are each stationary and jointly stationary, the sequences R_w /math> and R_ /math> known respectively as the autocorrelation of ''w'' 'n''and the cross-correlation between ''w'' 'n''and ''s'' 'n''can be defined as follows: :\begin R_w &= E\ \\ R_ &= E\ \end The derivative of the MSE may therefore be rewritten as: :\frac E \left \right_.html" ;"title="^2 \right ">^2 \right 2 \left ( \sum_^ R_w -ia_j \right ) - 2 R_ \qquad i = 0,\cdots, N. Note that for real w /math>, the autocorrelation is symmetric: R_w -i= R_w -j/math>Letting the derivative be equal to zero results in: :\sum_^N R_w -ia_j = R_ \qquad i = 0,\cdots, N. which can be rewritten (using the above symmetric property) in matrix form :\underbrace_ \underbrace_ = \underbrace_ These equations are known as the Wiener–Hopf equations. The matrix T appearing in the equation is a symmetric
Toeplitz matrix In linear algebra, a Toeplitz matrix or diagonal-constant matrix, named after Otto Toeplitz, is a matrix in which each descending diagonal from left to right is constant. For instance, the following matrix is a Toeplitz matrix: :\qquad\begin a & b ...
. Under suitable conditions on R, these matrices are known to be positive definite and therefore non-singular yielding a unique solution to the determination of the Wiener filter coefficient vector, \mathbf = \mathbf^\mathbf. Furthermore, there exists an efficient algorithm to solve such Wiener–Hopf equations known as the Levinson-Durbin algorithm so an explicit inversion of T is not required. In some articles, the cross correlation function is defined in the opposite way:R_ = E\Then, the \mathbf matrix will contain R_ \ldots R_ /math>; this is just a difference in notation. Whichever notation is used, note that for real w s /math>:R_ = R_ k/math>


Relationship to the least squares filter

The realization of the causal Wiener filter looks a lot like the solution to the
least squares The method of least squares is a standard approach in regression analysis to approximate the solution of overdetermined systems (sets of equations in which there are more equations than unknowns) by minimizing the sum of the squares of the re ...
estimate, except in the signal processing domain. The least squares solution, for input matrix \mathbf and output vector \mathbf is :\boldsymbol = (\mathbf ^\mathbf\mathbf)^\mathbf^\boldsymbol y . The FIR Wiener filter is related to the
least mean squares filter Least mean squares (LMS) algorithms are a class of adaptive filter used to mimic a desired filter by finding the filter coefficients that relate to producing the least mean square of the error signal (difference between the desired and the actual ...
, but minimizing the error criterion of the latter does not rely on cross-correlations or auto-correlations. Its solution converges to the Wiener filter solution.


Complex signals

For complex signals, the derivation of the complex Wiener filter is performed by minimizing E \left ^2_\right_]_=E_\left_ [n^*_\right_.html" ;"title=".html" ;"title="[n">[n^*_\right_">.html" ;"title="[n">[n^*_\right_/math>.__This_involves_computing_partial_derivatives_with_respect_to_both_the_real_and_imaginary_parts_of_a_i,_and_requiring_them_both_to_be_zero. The_resulting_Wiener-Hopf_equations_are: :\sum_^N_R_w_-ia_j^*_=_R___\qquad_i_=_0,\cdots,_N. which_can_be_rewritten_in_matrix_form: :\underbrace__\underbrace__=_\underbrace__ Note_here_that:\begin R_w[-k]_&=_R_w^*_\\ R__&=_R_^*[-k] \end The_Wiener_coefficient_vector_is_then_computed_as:\mathbf_=_^*


_Applications

The_Wiener_filter_has_a_variety_of_applications_in_signal_processing,_image_processing,_control_systems,_and_digital_communications.__These_applications_generally_fall_into_one_of_four_main_categories: *_
System_identification The field of system identification uses statistical methods to build mathematical models of dynamical systems from measured data. System identification also includes the optimal design of experiments for efficiently generating informative dat ...
*_
Deconvolution In mathematics, deconvolution is the operation inverse to convolution. Both operations are used in signal processing and image processing. For example, it may be possible to recover the original signal after a filter (convolution) by using a deco ...
_ *_
Noise_reduction Noise reduction is the process of removing noise from a signal. Noise reduction techniques exist for audio and images. Noise reduction algorithms may distort the signal to some degree. Noise rejection is the ability of a circuit to isolate an u ...
*_ Signal_detection For_example,_the_Wiener_filter_can_be_used_in_image_processing_to_remove_noise_from_a_picture._For_example,_using_the_Mathematica_function: WienerFilter mage,2/code>_on_the_first_image_on_the_right,_produces_the_filtered_image_below_it. It_is_commonly_used_to_denoise_audio_signals,_especially_speech,_as_a_preprocessor_before_
speech_recognition Speech recognition is an interdisciplinary subfield of computer science and computational linguistics that develops methodologies and technologies that enable the recognition and translation of spoken language into text by computers with the ...
.


__History_

The_filter_was_proposed_by_
Norbert_Wiener Norbert Wiener (November 26, 1894 – March 18, 1964) was an American mathematician and philosopher. He was a professor of mathematics at the Massachusetts Institute of Technology (MIT). A child prodigy, Wiener later became an early researcher ...
_during_the_1940s_and_published_in_1949.___The_discrete-time_equivalent_of_Wiener's_work_was_derived_independently_by_
Andrey_Kolmogorov Andrey Nikolaevich Kolmogorov ( rus, Андре́й Никола́евич Колмого́ров, p=ɐnˈdrʲej nʲɪkɐˈlajɪvʲɪtɕ kəlmɐˈɡorəf, a=Ru-Andrey Nikolaevich Kolmogorov.ogg, 25 April 1903 – 20 October 1987) was a Sovi ...
_and_published_in_1941.Kolmogorov_A.N:_'Stationary_sequences_in_Hilbert_space',_(In_Russian)_Bull._Moscow_Univ._1941_vol.2_no.6_1-40._English_translation_in_Kailath_T._(ed.)_''Linear_least_squares_estimation''_Dowden,_Hutchinson_&_Ross_1977__Hence_the_theory_is_often_called_the_''Wiener–Kolmogorov''_filtering_theory_(''cf.''_
Kriging In statistics, originally in geostatistics, kriging or Kriging, also known as Gaussian process regression, is a method of interpolation based on Gaussian process governed by prior covariances. Under suitable assumptions of the prior, kriging giv ...
).__The_Wiener_filter_was_the_first_statistically_designed_filter_to_be_proposed_and_subsequently_gave_rise_to_many_others_including_the_
Kalman_filter For statistics and control theory, Kalman filtering, also known as linear quadratic estimation (LQE), is an algorithm that uses a series of measurements observed over time, including statistical noise and other inaccuracies, and produces estima ...
.


_See_also

*__Wiener_deconvolution *_least_mean_squares_filter_ Least_mean_squares_(LMS)_algorithms_are_a_class_of_adaptive_filter_used_to_mimic_a_desired_filter_by_finding_the_filter_coefficients_that_relate_to_producing_the_least_mean_square_of_the_error_signal_(difference_between_the_desired_and_the_actual__...
*_ similarities_between_Wiener_and_LMS *_ linear_prediction *_ MMSE_estimator *_
Kalman_filter For statistics and control theory, Kalman filtering, also known as linear quadratic estimation (LQE), is an algorithm that uses a series of measurements observed over time, including statistical noise and other inaccuracies, and produces estima ...
*_ generalized_Wiener_filter *_
matched_filter In signal processing, a matched filter is obtained by correlating a known delayed signal, or ''template'', with an unknown signal to detect the presence of the template in the unknown signal. This is equivalent to convolving the unknown signal w ...
*_
Information_field_theory Information field theory (IFT) is a Bayesian statistical field theory relating to signal reconstruction, cosmography, and other related areas. IFT summarizes the information available on a physical field using Bayesian probabilities. It uses comput ...


_References


_Further_reading

*_
Thomas_Kailath Thomas Kailath (born June 7, 1935) is an electrical engineer, information theorist, control engineer, entrepreneur and the Hitachi America Professor of Engineering, Emeritus, at Stanford University. Professor Kailath has authored several books, i ...
,_
Ali_H._Sayed Ali H. Sayed (born Sao Paulo, Brazil, to parents of Lebanese people, Lebanese descent) is the dean of engineering at École Polytechnique Fédérale de Lausanne, EPFL (École polytechnique fédérale de Lausanne), where he teaches and conducts resea ...
,_and_
Babak_Hassibi Babak Hassibi ( fa, بابک حسیبی, born in Tehran, Iran) is an Iranian-American electrical engineer, computer scientist, and applied mathematician who is the inaugural Mose and Lillian S. Bohn Professor of Electrical Engineering and Compu ...
,_Linear_Estimation,_Prentice-Hall,_NJ,_2000,_.


_External_links

*Mathematic
WienerFilter
function {{DEFAULTSORT:Wiener_Filter Linear_filters Image_noise_reduction_techniques Signal_estimationhtml" ;"title="e ^2 \right ] =E \left [n^*_\right_.html" ;"title=".html" ;"title="[n">[n^* \right ">.html" ;"title="[n">[n^* \right /math>. This involves computing partial derivatives with respect to both the real and imaginary parts of a_i, and requiring them both to be zero. The resulting Wiener-Hopf equations are: :\sum_^N R_w -ia_j^* = R_ \qquad i = 0,\cdots, N. which can be rewritten in matrix form: :\underbrace_ \underbrace_ = \underbrace_ Note here that:\begin R_w[-k] &= R_w^* \\ R_ &= R_^*[-k] \end The Wiener coefficient vector is then computed as:\mathbf = ^*


Applications

The Wiener filter has a variety of applications in signal processing, image processing, control systems, and digital communications. These applications generally fall into one of four main categories: * System identification *
Deconvolution In mathematics, deconvolution is the operation inverse to convolution. Both operations are used in signal processing and image processing. For example, it may be possible to recover the original signal after a filter (convolution) by using a deco ...
*
Noise reduction Noise reduction is the process of removing noise from a signal. Noise reduction techniques exist for audio and images. Noise reduction algorithms may distort the signal to some degree. Noise rejection is the ability of a circuit to isolate an und ...
*
Signal detection Detection theory or signal detection theory is a means to measure the ability to differentiate between information-bearing patterns (called Stimulus (psychology), stimulus in living organisms, Signal (electronics), signal in machines) and random pa ...
For example, the Wiener filter can be used in image processing to remove noise from a picture. For example, using the Mathematica function: WienerFilter mage,2/code> on the first image on the right, produces the filtered image below it. It is commonly used to denoise audio signals, especially speech, as a preprocessor before
speech recognition Speech recognition is an interdisciplinary subfield of computer science and computational linguistics that develops methodologies and technologies that enable the recognition and translation of spoken language into text by computers with the m ...
.


History

The filter was proposed by
Norbert Wiener Norbert Wiener (November 26, 1894 – March 18, 1964) was an American mathematician and philosopher. He was a professor of mathematics at the Massachusetts Institute of Technology (MIT). A child prodigy, Wiener later became an early researcher i ...
during the 1940s and published in 1949. The discrete-time equivalent of Wiener's work was derived independently by
Andrey Kolmogorov Andrey Nikolaevich Kolmogorov ( rus, Андре́й Никола́евич Колмого́ров, p=ɐnˈdrʲej nʲɪkɐˈlajɪvʲɪtɕ kəlmɐˈɡorəf, a=Ru-Andrey Nikolaevich Kolmogorov.ogg, 25 April 1903 – 20 October 1987) was a Sovi ...
and published in 1941.Kolmogorov A.N: 'Stationary sequences in Hilbert space', (In Russian) Bull. Moscow Univ. 1941 vol.2 no.6 1-40. English translation in Kailath T. (ed.) ''Linear least squares estimation'' Dowden, Hutchinson & Ross 1977 Hence the theory is often called the ''Wiener–Kolmogorov'' filtering theory (''cf.''
Kriging In statistics, originally in geostatistics, kriging or Kriging, also known as Gaussian process regression, is a method of interpolation based on Gaussian process governed by prior covariances. Under suitable assumptions of the prior, kriging giv ...
). The Wiener filter was the first statistically designed filter to be proposed and subsequently gave rise to many others including the
Kalman filter For statistics and control theory, Kalman filtering, also known as linear quadratic estimation (LQE), is an algorithm that uses a series of measurements observed over time, including statistical noise and other inaccuracies, and produces estimat ...
.


See also

* Wiener deconvolution *
least mean squares filter Least mean squares (LMS) algorithms are a class of adaptive filter used to mimic a desired filter by finding the filter coefficients that relate to producing the least mean square of the error signal (difference between the desired and the actual ...
*
similarities between Wiener and LMS The Least mean squares filter solution converges to the Wiener filter solution, assuming that the unknown system is LTI and the noise is stationary. Both filters can be used to identify the impulse response of an unknown system, knowing only the ...
*
linear prediction Linear prediction is a mathematical operation where future values of a discrete-time signal are estimated as a linear function of previous samples. In digital signal processing, linear prediction is often called linear predictive coding (LPC) and ...
*
MMSE estimator In statistics and signal processing, a minimum mean square error (MMSE) estimator is an estimation method which minimizes the mean square error (MSE), which is a common measure of estimator quality, of the fitted values of a dependent variable. In ...
*
Kalman filter For statistics and control theory, Kalman filtering, also known as linear quadratic estimation (LQE), is an algorithm that uses a series of measurements observed over time, including statistical noise and other inaccuracies, and produces estimat ...
*
generalized Wiener filter The Wiener filter as originally proposed by Norbert Wiener is a signal processing filter which uses knowledge of the statistical properties of both the signal and the noise to reconstruct an optimal estimate of the signal from a noisy one-dimensio ...
*
matched filter In signal processing, a matched filter is obtained by correlating a known delayed signal, or ''template'', with an unknown signal to detect the presence of the template in the unknown signal. This is equivalent to convolving the unknown signal wi ...
*
Information field theory Information field theory (IFT) is a Bayesian statistical field theory relating to signal reconstruction, cosmography, and other related areas. IFT summarizes the information available on a physical field using Bayesian probabilities. It uses comput ...


References


Further reading

*
Thomas Kailath Thomas Kailath (born June 7, 1935) is an electrical engineer, information theorist, control engineer, entrepreneur and the Hitachi America Professor of Engineering, Emeritus, at Stanford University. Professor Kailath has authored several books, i ...
,
Ali H. Sayed Ali H. Sayed (born Sao Paulo, Brazil, to parents of Lebanese people, Lebanese descent) is the dean of engineering at École Polytechnique Fédérale de Lausanne, EPFL (École polytechnique fédérale de Lausanne), where he teaches and conducts resea ...
, and
Babak Hassibi Babak Hassibi ( fa, بابک حسیبی, born in Tehran, Iran) is an Iranian-American electrical engineer, computer scientist, and applied mathematician who is the inaugural Mose and Lillian S. Bohn Professor of Electrical Engineering and Compu ...
, Linear Estimation, Prentice-Hall, NJ, 2000, .


External links

*Mathematic
WienerFilter
function {{DEFAULTSORT:Wiener Filter Linear filters Image noise reduction techniques Signal estimation>e ^2 \right /math> =E \left [n^*_\right_.html" ;"title=".html" ;"title="[n">[n^* \right ">.html" ;"title="[n">[n^* \right /math>. This involves computing partial derivatives with respect to both the real and imaginary parts of a_i, and requiring them both to be zero. The resulting Wiener-Hopf equations are: :\sum_^N R_w -ia_j^* = R_ \qquad i = 0,\cdots, N. which can be rewritten in matrix form: :\underbrace_ \underbrace_ = \underbrace_ Note here that:\begin R_w[-k] &= R_w^* \\ R_ &= R_^*[-k] \end The Wiener coefficient vector is then computed as:\mathbf = ^*


Applications

The Wiener filter has a variety of applications in signal processing, image processing, control systems, and digital communications. These applications generally fall into one of four main categories: * System identification *
Deconvolution In mathematics, deconvolution is the operation inverse to convolution. Both operations are used in signal processing and image processing. For example, it may be possible to recover the original signal after a filter (convolution) by using a deco ...
*
Noise reduction Noise reduction is the process of removing noise from a signal. Noise reduction techniques exist for audio and images. Noise reduction algorithms may distort the signal to some degree. Noise rejection is the ability of a circuit to isolate an und ...
*
Signal detection Detection theory or signal detection theory is a means to measure the ability to differentiate between information-bearing patterns (called Stimulus (psychology), stimulus in living organisms, Signal (electronics), signal in machines) and random pa ...
For example, the Wiener filter can be used in image processing to remove noise from a picture. For example, using the Mathematica function: WienerFilter mage,2/code> on the first image on the right, produces the filtered image below it. It is commonly used to denoise audio signals, especially speech, as a preprocessor before
speech recognition Speech recognition is an interdisciplinary subfield of computer science and computational linguistics that develops methodologies and technologies that enable the recognition and translation of spoken language into text by computers with the m ...
.


History

The filter was proposed by
Norbert Wiener Norbert Wiener (November 26, 1894 – March 18, 1964) was an American mathematician and philosopher. He was a professor of mathematics at the Massachusetts Institute of Technology (MIT). A child prodigy, Wiener later became an early researcher i ...
during the 1940s and published in 1949. The discrete-time equivalent of Wiener's work was derived independently by
Andrey Kolmogorov Andrey Nikolaevich Kolmogorov ( rus, Андре́й Никола́евич Колмого́ров, p=ɐnˈdrʲej nʲɪkɐˈlajɪvʲɪtɕ kəlmɐˈɡorəf, a=Ru-Andrey Nikolaevich Kolmogorov.ogg, 25 April 1903 – 20 October 1987) was a Sovi ...
and published in 1941.Kolmogorov A.N: 'Stationary sequences in Hilbert space', (In Russian) Bull. Moscow Univ. 1941 vol.2 no.6 1-40. English translation in Kailath T. (ed.) ''Linear least squares estimation'' Dowden, Hutchinson & Ross 1977 Hence the theory is often called the ''Wiener–Kolmogorov'' filtering theory (''cf.''
Kriging In statistics, originally in geostatistics, kriging or Kriging, also known as Gaussian process regression, is a method of interpolation based on Gaussian process governed by prior covariances. Under suitable assumptions of the prior, kriging giv ...
). The Wiener filter was the first statistically designed filter to be proposed and subsequently gave rise to many others including the
Kalman filter For statistics and control theory, Kalman filtering, also known as linear quadratic estimation (LQE), is an algorithm that uses a series of measurements observed over time, including statistical noise and other inaccuracies, and produces estimat ...
.


See also

* Wiener deconvolution *
least mean squares filter Least mean squares (LMS) algorithms are a class of adaptive filter used to mimic a desired filter by finding the filter coefficients that relate to producing the least mean square of the error signal (difference between the desired and the actual ...
*
similarities between Wiener and LMS The Least mean squares filter solution converges to the Wiener filter solution, assuming that the unknown system is LTI and the noise is stationary. Both filters can be used to identify the impulse response of an unknown system, knowing only the ...
*
linear prediction Linear prediction is a mathematical operation where future values of a discrete-time signal are estimated as a linear function of previous samples. In digital signal processing, linear prediction is often called linear predictive coding (LPC) and ...
*
MMSE estimator In statistics and signal processing, a minimum mean square error (MMSE) estimator is an estimation method which minimizes the mean square error (MSE), which is a common measure of estimator quality, of the fitted values of a dependent variable. In ...
*
Kalman filter For statistics and control theory, Kalman filtering, also known as linear quadratic estimation (LQE), is an algorithm that uses a series of measurements observed over time, including statistical noise and other inaccuracies, and produces estimat ...
*
generalized Wiener filter The Wiener filter as originally proposed by Norbert Wiener is a signal processing filter which uses knowledge of the statistical properties of both the signal and the noise to reconstruct an optimal estimate of the signal from a noisy one-dimensio ...
*
matched filter In signal processing, a matched filter is obtained by correlating a known delayed signal, or ''template'', with an unknown signal to detect the presence of the template in the unknown signal. This is equivalent to convolving the unknown signal wi ...
*
Information field theory Information field theory (IFT) is a Bayesian statistical field theory relating to signal reconstruction, cosmography, and other related areas. IFT summarizes the information available on a physical field using Bayesian probabilities. It uses comput ...


References


Further reading

*
Thomas Kailath Thomas Kailath (born June 7, 1935) is an electrical engineer, information theorist, control engineer, entrepreneur and the Hitachi America Professor of Engineering, Emeritus, at Stanford University. Professor Kailath has authored several books, i ...
,
Ali H. Sayed Ali H. Sayed (born Sao Paulo, Brazil, to parents of Lebanese people, Lebanese descent) is the dean of engineering at École Polytechnique Fédérale de Lausanne, EPFL (École polytechnique fédérale de Lausanne), where he teaches and conducts resea ...
, and
Babak Hassibi Babak Hassibi ( fa, بابک حسیبی, born in Tehran, Iran) is an Iranian-American electrical engineer, computer scientist, and applied mathematician who is the inaugural Mose and Lillian S. Bohn Professor of Electrical Engineering and Compu ...
, Linear Estimation, Prentice-Hall, NJ, 2000, .


External links

*Mathematic
WienerFilter
function {{DEFAULTSORT:Wiener Filter Linear filters Image noise reduction techniques Signal estimation